DOCUMENT RESUME 



ED 443 831 



TM 031 313 



AUTHOR 

TITLE 



PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Haro, Analee; Yoon, Ruth 

The Los Angeles Annenberg Metropolitan Project: n How Can 
Internal and External Evaluations Help Improve Parent 
Involvement Programs ? " 

2000-04-00 

13p . ; Paper presented at the Annual Meeting of the American 
Educational Research Association (New Orleans, LA, April 
24-28, 2000) . 

Reports - Evaluative (142) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

Educational Change; Elementary Secondary Education; 
♦Evaluation Methods; Evaluation Utilization; *Parent 
Participation; Program Evaluation; Urban Schools 
Annenberg Foundation; *External Evaluation; * Internal 
Evaluation; Los Angeles City Schools CA 



ABSTRACT 



The Los Angeles Annenberg Metropolitan Project (LAAMP) has 
several initiatives, including Parents as Learning Partners (PLP) , that 
attempt to assist parents and schools to work collaboratively in order to 
increase student achievement. Using PLP, one of LAAMP ' s major parent 
engagement initiatives, as a model, this paper discusses using both internal 
and external evaluations to provide a better understanding of how to 
implement parent involvement programs . An external evaluation team has been 
hired by LAAMP to evaluate PLP through a longitudinal and quasi -experimental 
design to track parent-child pairs over time. Six PLP and six non- PLP schools 
were compared to determine the impact of parent involvement . The internal 
evaluation of PLP was carried out by LAAMP in collaboration with the schools 
and school families. Advantages and weaknesses of both internal and external 
evaluations are discussed. It is important to use both types of evaluation to 
evaluate and continue implementing a parent involvement program. (SLD) 
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Introduction 

It has long been established that children whose parents are involved in their education 
experience more academic success. With this in mind, the Los Angeles Annenberg Metropolitan 
Project (LAAMP) has several initiatives, including Parents As Learning Partners, aimed at 
assisting parents and schools to work collaboratively in order to increase student achievement. 

These initiatives use as a theoretical framework Joyce Epstein’s research 1 on 
home/school partnerships. Epstein outlines key elements in partnership-building: parenting, 
communicating, volunteering, learning at home, decision making and collaborating with the 
community. When parents are supporting academics in the home and in the school, a partnership 
is formed between the parents and the school that provides a supportive environment where 
children can succeed. 

LAAMP’s parent engagement initiatives seek to improve academic achievement through 
the creation of these partnerships. The initiatives focus on utilizing Epstein’s six key elements to 
more effectively create a partnership between the teacher and school. In addition to 
implementing these initiatives, LAAMP has and continues to evaluate these programs, both 
internally and externally. These evaluations determine the quality of program implementation, 
areas needing improvement, and what assistance and support is needed. 

Using Parents as Learning Partners, one of LAAMP’s major parent engagement 
initiatives 2 , as a model, this paper discusses how utilizing both internal and external evaluations 



1 See Epstein, J.L., Coates, K. C., Salinas, M. G., and B. S. Simon (1997) School, Family and Community 
Partnerships: Your Handbook for Action. Corwin Press.; Epstein, J.L. (1995) School/family/ community 
partnerships: Caring for the children we share. Phi Delta Kappan, 76, 701-712. 

" LAAMP has another major parent engagement initiative, Families in Schools. It is a collaboration between 
LAAMP and the Parent Institute for Quality Education. Twenty-nine schools in three school districts participated in 
a nine-week parent education course. Course graduates then participated in a four month coaching component 



can provide a more complete understanding of how to best implement parent involvement 
programs. Utilizing both evaluation strategies is particularly important to parent involvement 
because of the numerous and diverse stakeholders that are needed to create a successful school- 
home partnership. Both evaluations, internal and external, have their strengths and limitations. 
Overall, having both evaluations occurring concurrently and communicating continuously with 
all evaluators can provide organizations and schools with a powerful tool to improve academic 
achievement through improved home/school partnerships. 

Parents as Learning Partners (PLP) 

The Parents as Learning Partners grant was awarded to 33 schools in three school 
families 3 in Los Angeles County. The five-year grant from LAAMP was funded by the Weingart 
Foundation, Annenberg Foundation, and the participating school districts and school sites. PLP 
focuses primarily on three of Epstein’s key elements: communicating, parenting and learning at 
home. 

PLP aims to improve these areas through professional development for teachers, parent 
education and other school services. The professional development for teachers focuses on how 
to better communicate with parents, how to incoiporate volunteers in the classroom and utilizing 
homework that involves both the child and the parent (interactive homework). The parent 
education focuses on communication with teachers and providing learning support at home. The 
grant also provides school services including personnel to assist with outreach and provide 
parents with an additional source of support and communication with the school; lending 
libraries to encourage parents and children to increase the amount of reading in the home; child 



where parents were encouraged to practice behaviors that contribute to higher student academic achievement. See 
Johnson, Deborah and Jiang, Ying Hong (2000). Families In Schools: How Did a Parent Education 
Program Change Parent Behaviors Related to Student Achievement? New Orleans: Paper presented at the Annual 
Meeting of the American Educational Research Association. 

J A “school family” consists of the elementary, middle and high schools in a feeder pattern or neighborhood. 



care and translation services to facilitate parent participation in workshops; volunteering and 
other school activities. 

PLP also works with each school to implement and support the school action team. The 
action team is comprised of administrators, teachers, staff, parents, parent ambassadors 4 and 
community representatives who work to facilitate partnerships in the school. The action team is 
responsible for creating an action plan, which details the parent involvement efforts at the school. 
The action team works to determine what parents, teachers and the school, as a whole, needs in 
order to improve the home/school partnerships. 

Each school family, a collection of schools that follow the feeder pattern from elementary 
to high school, has a full-time facilitator funded by the grant who works to provide guidance and 
technical support. The School Family Facilitator assists each action team and works with the 
school to provide professional development, parent education and other services. 

External Evaluation 

An external research team has been contracted by LAAMP to evaluate PLP. This 
evaluation integrates a longitudinal and quasi-experimental design to track parent-child pairs 
over time. Six PLP and six matching non- PLP schools are compared to determine the impact of 
PLP on parent involvement, teacher participation and student achievement . 5 

The evaluation revealed some significant findings. First, a larger number of PLP teachers 
participated in professional development in the area of parent involvement although it was 
mostly informal. Nevertheless, teachers did not feel prepared to engage parents. Teachers also 
reported that parent involvement, such as classroom visits and volunteering, was low. Even 

4 Each school family utilized a different name for community members hired to work on parent involvement. These 
groups of people were called Parent Ambassadors, Parent Mentors and Community Representatives. Even though 
they held different titles, their work was essentially the same 

5 For a more complete description of the external evaluation, see Quigley, Denise D. (2000) Parents and Teachers 
Working Together to Support Third Grade Achievement: Parents as Learning Partners (PLP) Findings. New 
Orleans: Paper presented at the Annual Meeting of the American Educational Research Association. 



5 

though PLP has invested a large amount of money on installing phone lines, few parents were 
receiving calls from teachers. 

Parents report being more aware of parent involvement activities but there has not been 
an increase in parent participation at these events. The evaluation also found the student 
achievement at PLP schools is slightly better in reading than in non-PLP in one school district. 

No differences were found in math and language arts. 

Based on this information, PLP has several plans to improve teacher-parent 
communication. There continues to be a need for more professional development since teachers 
do not feel prepared to engage parents, even though PLP teachers reported participating more in 
professional development than non-PLP teachers. PLP will offer more professional development 
for teachers. In addition, a teacher training curriculum in parent involvement is being developed 
in conjunction with California State University, the largest teacher-training institution in 
California, to assist new teachers in this area. PLP is also redefining the role of the PLP 
facilitators in order to increase their capacity to assist school action teams and provide 
professional development to teachers. 

PLP Internal Evaluation 

At the end of 1998-1999 academic year and continuing into the 1999-2000 academic year, 
PLP’s third year, LAAMP in collaboration with the schools and school families conducted an 
internal evaluation of PLP. The evaluation consisted of day-long site visits to each school. All 
stakeholders, including parents, teachers, principals, parent ambassadors, teacher’s assistants, 
coordinators and action teams, were interviewed in focus groups that lasted from thirty minutes 
to an hour. In addition, the site visit team reviewed documentation such as meeting agendas, 
parent workshop schedules and sign-in sheets, etc. There were two goals for these visits: 

1. What challenges is the school facing in creating school/home partnerships to 
improve academic achievement? 
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2. What areas of need are common to PLP schools and what actions does the project 
staff need to take to meet the needs of the schools? 

What challenges is the school facing in creating school/home partnerships to improve academic 
achievement? 

By conducting an internal evaluation, it was possible to focus not only on the areas of 
need that were found in most of the schools but also to learn about issues that were particular to a 
given school. Individual school sites have different challenges and needs and by visiting each 
school, these challenges emerged. For instance, one school did not have the space to house the 
Parent Center and had to utilize the auditorium to conduct workshops and other programs. As a 
result, few workshops were being offered which made it more difficult for PLP parent 
ambassadors to reach parents. Another school had not been able to hire a parent ambassador and 
the teachers were shouldering the responsibility of implementing PLP. 

These visits also provided examples of creative ways that schools were reaching out to 
parents. Several schools found that their parents were most available on the weekends and began 
offering Saturday workshops with great success. Another school had a reading event during the 
day where parents came to school with their children and participated in reading activities. As a 
reward for their participation, the school gave the parents two books to take home. This event 
was very successful and attracted many parents who spent the morning reading with their 
children. 

Schools also found innovative ways to increase parent-teacher communication and to 
inform parents of upcoming events. One school has “Breakfast with the Principal” each 
Thursday. This provides parents with an opportunity become better acquainted with the 
principal and to begin a dialogue about how to better support children’s learning. The principal 
utilizes this time to discuss ways parents could support their children and what programs are 
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available for parents. “Coffee and Conversation” provides parents and teachers with an informal 
opportunity to interact. In an another school, parent volunteers bring coffee and cookies to the 
front of the school in the mornings. Parents can stop by the table and have a cup of coffee and 
learn about upcoming events and activities at the school. 

The visits also illuminated the challenges that the school had not yet noticed. At one 
elementary school, instead of assigning homework daily, the teachers were sending homework 
packets each Monday which were to be completed and returned each Friday. By doing so, the 
teacher wanted to give students and parents more freedom in deciding when to do then- 
homework. In this way, if students had a soccer game or the family had another event on a given 
evening, they could do the homework on other nights. The teachers also felt that the packets 
taught the students to be responsible for pacing themselves and utilized it as another way to teach 
discipline. Without being asked about the homework packets, parents expressed concern about 
them during the focus group. They were upset that teachers were not instilling discipline in their 
children because they were not assigning homework daily. Instead of finding the packets to be 
positive, the parents were upset that they were being utilized. During the site visit, the school 
learned that there was a need for further communication between parents and teachers about 
homework packets so that both parents and teachers were working towards the same goal of 
developing positive habits regarding homework completion in the children. 

The site visits also provided an opportunity to determine how much support school site 
administrators were giving PLP. Some administrators were actively involved in implementing 
their action plan. Other administrators seemed to agree that more needed to be done but did not 
take steps that would further partnerships at the school. In order to support both those 
administrators who were actively working with PLP and especially those who were more 
marginally involved, PLP hired a principal who recently retired from a PLP school to provide 
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coaching and technical assistance to the administrators and the action teams at individual school 
sites. The retired principal understands the many demands and challenges facing a school and 
can assist each school to find creative solutions. 

What areas of need are common to PLP schools and what actions does the project staff need to 
take to meet the needs of the schools? 

In addition to individual differences, visiting all the school sites pointed to trends that 
were found across most schools. These trends included common successes such as successful 
workshops. For example, schools found that workshops where parents created items such as 
flashcards or cassettes that could then be utilized with their children at home had been very 
successful. Similarly, a fundamental part of PLP has been to employ community members to 
work as parent ambassadors for the school. The parent ambassadors are members of the action 
team and work to improve outreach to parents. In addition, parent ambassadors provide 
workshops and coordinate childcare during events. All schools found that the ambassadors were 
significantly improving parent involvement and their outreach was very effective. In addition, it 
was found that they had also experienced personal growth. 

There were also challenges that most the schools are facing. For example, effective 
implementation of the school action team was one of the biggest challenges facing schools. 
Schools were at different stages in the development of an action team ranging from not having an 
action team at all to one having consistent members and regular meetings. Some schools made 
the action team one of the standing committees at the school. Other “action teams” merely 
consisted of an administrator and the community representative. Most action teams found it 
challenging to recruit and integrate parents into the team. It was clear that all action teams 
needed more guidance and support in becoming a strong, permanent part of the school. As a 
result, PLP facilitators and the retired principal will spend more time working directly with 
action teams. The facilitators will work with school action teams to evaluate which elements of 
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their action plans have been most effective and what aspect need improvement. In addition, each 
school will create a three-year action plan that will guide the work for the final two years of the 
grant as well as how the school plans to sustain its partnership efforts after PLP funds end. 

Similarly, even though schools had had varying degrees of success reaching out to 
parents, all schools were still in need of ideas about how to reach out to an even larger number of 
parents. Even though Parent Ambassadors were reaching more parents, the numbers overall 
were still low. Parent workshops often had twelve to fifteen parents in a school with at least 600 
students. Few strategies have been implemented to reach parents who do not attend activities at 
the school site. PLP is creating a “Best Practices Booklet” to share ideas among the schools 
about effective strategies, such as those described above. 

Communication between teachers and parents needed to improve. It was infrequent and 
the quality of the conversations was often poor. Teachers did not know how to best utilize 
parents in the classroom and communicated with parents only when problems arose with the 
student. Most teachers relied only on Back to School Night, parent-teacher conferences and 
Open House to communicate with parents. The internal evaluation revealed the need for more 
professional development in the area of parent involvement and partnership building. Among 
topics that would be covered included making positive phone calls about students; calling to 
welcome parents and children at the beginning of the school year and conducting effective 
parent-teacher conferences. 

The internal evaluation provided the PLP staff with an opportunity to take an “inside” 
look at schools in order to determine what steps PLP needs to take at individual schools and 
across school sites. It was a valuable tool to find areas of strength and challenges that may 
otherwise not have been obvious. The evaluation provided the schools and school families with 
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recommendations that will inform their work in the coming years. It was also an opportunity to 
provide schools with promising practices that had been successful at other schools. 

Even though the internal evaluation was extremely successful, there were also limitations 
to this type of evaluation. Parents commented that their children were experiencing more 
success, however it was not possible to determine whether academic achievement was being 
affected by PLP. In addition, focus groups were not randomly selected. In some cases the 
people who attended were those that were most knowledgeable or most supportive of the work of 
PLP. For instance, sometimes the parents that attended the focus group were the parents who 
spent the most time at the Parent Center or who volunteered often. Although the schools were 
asked to invite a cross-section of people for the focus groups and interviews, this did not occur. 
Therefore, it was not possible to determine if the views expressed by those participating in the 
focus groups was representative of the rest of the school. Furthermore, it was not possible to 
determine which of the many activities that were occurring at a given school were most likely to 
support student achievement. 

Internal vs. External Evaluations 

In deciding between an internal or external evaluation, organizations and schools need to 
consider factors such as time, budget and reasons for the evaluation. Depending on the answers 
to these questions an internal or an external evaluation may be more appropriate. 

Internal Evaluation 

Internal evaluations have several key advantages. They are more timely. Immediately 
the program has information that begins informing their decisions. Due to the nature of the PLP 
External Evaluation, it took several months before LAAMP had any results about the program. 
Similarly, programs can immediately make changes to the program. 
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As was stated earlier, when evaluating a program that is in more than one school site, 
internal evaluations can reveal school specific challenges or successes. The PLP site visits 
revealed many very specific challenges facing an individual school. 

Internal evaluations also have the buy-in of the people at the school site because they are 
part of the evaluation. Each school wanted to contribute to the evaluation and were excited 
about the opportunity to have their voices heard. 

On the other hand, internal evaluation requires staff time. School Family Facilitators, 
LAAMP and district personnel spent a considerable number of days conducting the site visits for 
PLP. School sites were also asked to take time to host the visit. All of this time is taken away 
from other responsibilities and activities that could be done. Similarly, some areas cannot 
always be addressed by internal evaluations. The fact that those involved are evaluating the 
program creates for some bias that, may prevent certain information from being found. 

External evaluations 

External evaluation can provide a more objective view of the program. External 
evaluators, who are not involved with the day to day operations of the program, may consider 
factors and methodologies that should be measured that may otherwise be overlooked. The 
external evaluation of PLP focused on following parent-child pairs to determine if the work of 
PLP is contributing to student achievement. This type of evaluation is beyond the scope of what 
the personnel involved with PLP would be able to do. As part of the evaluation, the effect of 
PLP on student achievement provides important information that necessary to determine the 
areas of success for PLP. 

External evaluations also offer more credibility for people outside of the program. 

Funders, parents, community members, and the public at large view external evaluations as more 
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objective than those conducted by internal program staff. In addition, when programs achieve 
positive outcomes, an external research team can add credibility to the findings. 

However, external evaluations can be more costly and take more time. In addition, it is 
difficult to control the factors that will be examined by outside evaluators. 

Conclusion 

Both internal and external evaluations have strengths and limitations. Participating in 
both types of evaluations, it has been come clear that it is important to utilize both in order to 
successfully evaluate and continue implementing a parent involvement program. Given the 
different people that are needed to implement a parent involvement program successfully, it is 
important to evaluate as many facets of the program as possible. At the same time, program 
funders and organizations also need to provide measures of success. As such a combination of 
internal and external evaluations can provide the most reliable and thorough understanding of 
any parent involvement program. 
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